Evolutionary Optimization of Least-Squares Support Vector Machines

The performance of Kernel Machines depends to a large extent on its kernel function and hyperparameters. Selecting these is traditionally done using intuition or a costly “trial-and-error” approach, which typically prevents these methods from being used to their fullest extent. Therefore, two automated approaches are presented for the selection of a suitable kernel function and optimal hyperparameters for the Least-Squares Support Vector Machine. The first approach uses Evolution Strategies, Genetic Algorithms, and Genetic Algorithms with floating point representation to find optimal hyperparameters in a timely manner. On benchmark data sets the standard Genetic Algorithms approach outperforms the two other evolutionary algorithms and is shown to be more efficient than grid search. The second approach aims to improve the generalization capacity of the machine by evolving combined kernel functions using Genetic Programming. Empirical studies show that this model indeed increases the generalization performance of the machine, although this improvement comes at a high computational cost. This suggests that the approach may be justified primarily in applications where prediction errors can have severe consequences, such as in medical settings. Arjan Gijsberts Italian Institute of Technology, Via Morego, 30 – Genoa 16163, Italy Delft University of Technology, Mekelweg 4, 2628 CD Delft, The Netherlands e-mail: [email protected] Giorgio Metta Italian Institute of Technology, Via Morego, 30 – Genoa 16163, Italy University of Genoa, Viale F. Causa, 13 – Genoa 16145, Italy e-mail: [email protected] Léon Rothkrantz Delft University of Technology, Mekelweg 4, 2628 CD Delft, The Netherlands Netherlands Defence Academy, Nieuwe Diep 8, 1781 AT Den Helder, The Netherlands e-mail: [email protected]